Whole-book recognition.
Identifieur interne : 000237 ( Main/Exploration ); précédent : 000236; suivant : 000238Whole-book recognition.
Auteurs : Pingping Xiu [États-Unis] ; Henry S. BairdSource :
- IEEE transactions on pattern analysis and machine intelligence [ 1939-3539 ] ; 2012.
Abstract
Whole-book recognition is a document image analysis strategy that operates on the complete set of a book's page images using automatic adaptation to improve accuracy. We describe an algorithm which expects to be initialized with approximate iconic and linguistic models--derived from (generally errorful) OCR results and (generally imperfect) dictionaries--and then, guided entirely by evidence internal to the test set, corrects the models which, in turn, yields higher recognition accuracy. The iconic model describes image formation and determines the behavior of a character-image classifier, and the linguistic model describes word-occurrence probabilities. Our algorithm detects "disagreements" between these two models by measuring cross entropy between 1) the posterior probability distribution of character classes (the recognition results resulting from image classification alone) and 2) the posterior probability distribution of word classes (the recognition results from image classification combined with linguistic constraints). We show how disagreements can identify candidates for model corrections at both the character and word levels. Some model corrections will reduce the error rate over the whole book, and these can be identified by comparing model disagreements, summed across the whole book, before and after the correction is applied. Experiments on passages up to 180 pages long show that when a candidate model adaptation reduces whole-book disagreement, it is also likely to correct recognition errors. Also, the longer the passage operated on by the algorithm, the more reliable this adaptation policy becomes, and the lower the error rate achieved. The best results occur when both the iconic and linguistic models mutually correct one another. We have observed recognition error rates driven down by nearly an order of magnitude fully automatically without supervision (or indeed without any user intervention or interaction). Improvement is nearly monotonic, and asymptotic accuracy is stable, even over long runs. If implemented naively, the algorithm runs in time quadratic in the length of the book, but random subsampling and caching techniques speed it up by two orders of magnitude with negligible loss of accuracy. Whole-book recognition has potential applications in digital libraries as a safe unsupervised anytime algorithm.
DOI: 10.1109/TPAMI.2012.50
PubMed: 22350165
Affiliations:
Links toward previous steps (curation, corpus...)
- to stream PubMed, to step Corpus: 000031
- to stream PubMed, to step Curation: 000031
- to stream PubMed, to step Checkpoint: 000031
- to stream Ncbi, to step Merge: 000122
- to stream Ncbi, to step Curation: 000122
- to stream Ncbi, to step Checkpoint: 000122
- to stream Main, to step Merge: 000240
- to stream Main, to step Curation: 000237
Le document en format XML
<record><TEI><teiHeader><fileDesc><titleStmt><title xml:lang="en">Whole-book recognition.</title><author><name sortKey="Xiu, Pingping" sort="Xiu, Pingping" uniqKey="Xiu P" first="Pingping" last="Xiu">Pingping Xiu</name><affiliation wicri:level="2"><nlm:affiliation>Microsoft Advertising R&D, One Microsoft Way, Redmond, WA 98052-6399, USA. pingxiu@imdb.com</nlm:affiliation><country xml:lang="fr">États-Unis</country><wicri:regionArea>Microsoft Advertising R&D, One Microsoft Way, Redmond, WA 98052-6399</wicri:regionArea><placeName><region type="state">Washington (État)</region></placeName></affiliation></author><author><name sortKey="Baird, Henry S" sort="Baird, Henry S" uniqKey="Baird H" first="Henry S" last="Baird">Henry S. Baird</name></author></titleStmt><publicationStmt><idno type="wicri:source">PubMed</idno><date when="2012">2012</date><idno type="doi">10.1109/TPAMI.2012.50</idno><idno type="RBID">pubmed:22350165</idno><idno type="pmid">22350165</idno><idno type="wicri:Area/PubMed/Corpus">000031</idno><idno type="wicri:Area/PubMed/Curation">000031</idno><idno type="wicri:Area/PubMed/Checkpoint">000031</idno><idno type="wicri:Area/Ncbi/Merge">000122</idno><idno type="wicri:Area/Ncbi/Curation">000122</idno><idno type="wicri:Area/Ncbi/Checkpoint">000122</idno><idno type="wicri:Area/Main/Merge">000240</idno><idno type="wicri:Area/Main/Curation">000237</idno><idno type="wicri:Area/Main/Exploration">000237</idno></publicationStmt><sourceDesc><biblStruct><analytic><title xml:lang="en">Whole-book recognition.</title><author><name sortKey="Xiu, Pingping" sort="Xiu, Pingping" uniqKey="Xiu P" first="Pingping" last="Xiu">Pingping Xiu</name><affiliation wicri:level="2"><nlm:affiliation>Microsoft Advertising R&D, One Microsoft Way, Redmond, WA 98052-6399, USA. pingxiu@imdb.com</nlm:affiliation><country xml:lang="fr">États-Unis</country><wicri:regionArea>Microsoft Advertising R&D, One Microsoft Way, Redmond, WA 98052-6399</wicri:regionArea><placeName><region type="state">Washington (État)</region></placeName></affiliation></author><author><name sortKey="Baird, Henry S" sort="Baird, Henry S" uniqKey="Baird H" first="Henry S" last="Baird">Henry S. Baird</name></author></analytic><series><title level="j">IEEE transactions on pattern analysis and machine intelligence</title><idno type="eISSN">1939-3539</idno><imprint><date when="2012" type="published">2012</date></imprint></series></biblStruct></sourceDesc></fileDesc><profileDesc><textClass></textClass></profileDesc></teiHeader><front><div type="abstract" xml:lang="en">Whole-book recognition is a document image analysis strategy that operates on the complete set of a book's page images using automatic adaptation to improve accuracy. We describe an algorithm which expects to be initialized with approximate iconic and linguistic models--derived from (generally errorful) OCR results and (generally imperfect) dictionaries--and then, guided entirely by evidence internal to the test set, corrects the models which, in turn, yields higher recognition accuracy. The iconic model describes image formation and determines the behavior of a character-image classifier, and the linguistic model describes word-occurrence probabilities. Our algorithm detects "disagreements" between these two models by measuring cross entropy between 1) the posterior probability distribution of character classes (the recognition results resulting from image classification alone) and 2) the posterior probability distribution of word classes (the recognition results from image classification combined with linguistic constraints). We show how disagreements can identify candidates for model corrections at both the character and word levels. Some model corrections will reduce the error rate over the whole book, and these can be identified by comparing model disagreements, summed across the whole book, before and after the correction is applied. Experiments on passages up to 180 pages long show that when a candidate model adaptation reduces whole-book disagreement, it is also likely to correct recognition errors. Also, the longer the passage operated on by the algorithm, the more reliable this adaptation policy becomes, and the lower the error rate achieved. The best results occur when both the iconic and linguistic models mutually correct one another. We have observed recognition error rates driven down by nearly an order of magnitude fully automatically without supervision (or indeed without any user intervention or interaction). Improvement is nearly monotonic, and asymptotic accuracy is stable, even over long runs. If implemented naively, the algorithm runs in time quadratic in the length of the book, but random subsampling and caching techniques speed it up by two orders of magnitude with negligible loss of accuracy. Whole-book recognition has potential applications in digital libraries as a safe unsupervised anytime algorithm.</div></front></TEI><affiliations><list><country><li>États-Unis</li></country><region><li>Washington (État)</li></region></list><tree><noCountry><name sortKey="Baird, Henry S" sort="Baird, Henry S" uniqKey="Baird H" first="Henry S" last="Baird">Henry S. Baird</name></noCountry><country name="États-Unis"><region name="Washington (État)"><name sortKey="Xiu, Pingping" sort="Xiu, Pingping" uniqKey="Xiu P" first="Pingping" last="Xiu">Pingping Xiu</name></region></country></tree></affiliations></record>Pour manipuler ce document sous Unix (Dilib)
EXPLOR_STEP=$WICRI_ROOT/Ticri/CIDE/explor/OcrV1/Data/Main/Exploration
HfdSelect -h $EXPLOR_STEP/biblio.hfd -nk 000237 | SxmlIndent | more
Ou
HfdSelect -h $EXPLOR_AREA/Data/Main/Exploration/biblio.hfd -nk 000237 | SxmlIndent | more
Pour mettre un lien sur cette page dans le réseau Wicri
{{Explor lien
|wiki= Ticri/CIDE
|area= OcrV1
|flux= Main
|étape= Exploration
|type= RBID
|clé= pubmed:22350165
|texte= Whole-book recognition.
}}
Pour générer des pages wiki
HfdIndexSelect -h $EXPLOR_AREA/Data/Main/Exploration/RBID.i -Sk "pubmed:22350165" \
| HfdSelect -Kh $EXPLOR_AREA/Data/Main/Exploration/biblio.hfd \
| NlmPubMed2Wicri -a OcrV1
| This area was generated with Dilib version V0.6.32. |  |